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Abstract. Many natural and engineered complex networks have intricate mesoscopic organiza- 
tion, e.g., the clustering of the constituent nodes into several communities or modules. Often, such 
modularity is manifested at several different hierarchical levels, where the clusters defined at one 
level appear as elementary entities at the next higher level. Using a simple model of a hierarchical 
modular network, we show that such a topological structure gives rise to characteristic time-scale 
separation between dynamics occurring at different levels of the hierarchy. This generalizes our 
earlier result for simple modular networks, where fast intra-modular and slow inter-modular pro- 
cesses were clearly distinguished. Investigating the process of synchronization of oscillators in a 
hierarchical modular network, we show the existence of as many distinct time-scales as there are 
hierarchical levels in the system. This suggests a possible functional role of such mesoscopic or- 
ganization principle in natural systems, viz., in the dynamical separation of events occurring at 
different spatial scales. 
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1. Introduction 

Complex networks are ubiquitous fP] . They are seen in phenomena studied by physicists 
from resistor networks to polymer contact structure to spin interactions in disordered sys- 
tems in engineered systems (e.g., the internet and the power grid) and in the biologi- 
cal world from the intra-cellular signaling system to neuronal networks to ecological food 
webs lit]. Analysis of such networks can lead not only to fundamental understanding of 
how complex systems function, but may also have immediate practical relevance. For 
example, understanding the network of intra-cellular communication leading to identifi- 
cation of the key players can result in developing drugs targeted specifically towards these 
molecules i^. On a much larger scale, by revealing how the contact structure among sus- 
ceptible and infected individuals govern the propagation of epidemics, the manipulation 
of networks may play a significant role in public health. 

Beginning in 1998-99, over the last decade interest in complex networks has grown 
by leaps and bounds among physicists. However, the focus of research activity related 
to such systems has been on either the microscopic properties of individual nodes such 
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as the distribution of degree (i.e., the number of links for a node) or the macroscopic 
properties characterizing the entire network in terms of a global value, such as average 
path length or clustering coefficient |6]. Recent research, on the other hand, has revealed 
that networks which are indistinguishable at either of these two scales may nevertheless 
have radically different behavior |7]. The origin of this difference lies in their mesoscopic 
organization which can be structurally manifested as patterns in the arrangement of links 
between subparts of the network . 

One of the prominent examples of such organizing principles operating in networks is 
the existence of communities, also referred to as modularity |9]. Modules can be defined 
as subnetworks whose components are much more densely and/or strongly connected 
with each other as compared to connections with components that belong to different 
subnetworks. In earlier work we have shown that existence of modules can give rise to 
two distinct time-scales for the dynamics on such networks: a strongly modular character 
of the system, as indicated by a low value for the ratio of within-module to between- 
module connections, results in a sharp distinction between fast intra-modular and slower 
inter-modular processes iQl- Another mesoscopic organizing principle observed in many 
networks is hierarchy, i.e., an arrangement in which entities are ordered in several levels 
or layers 1.10.1 . We are most familiar with hierarchy as seen in social organization; how- 
ever, they are also present in other systems, as for example in the neuronal communication 
network which has a clearly defined direction of information flow, beginning with the sen- 
sory organs and culminating with motor response. While both modular and hierarchical 
features are increasingly being reported in a wide range of systems, it is still not clear 
what role such patterns play in the dynamics and function of the corresponding systems. 
Thus, the second wave of research in complex networks that is just beginning focuses on 
these mesoscopic aspects of networks. 

There can be even more complicated intermediate-scale features in networks, in partic- 
ular, hierarchically nested modularity, where modules and meta-modules (composed of 
multiple strongly connected modules) may occur at different hierarchical levels. Such hi- 
erarchical modularity has been observed in many naturally occurring systems, including 
biological metabolic networks jl ill , ecological food webs_n2J, social groups [13, 14], fi- 



nancial markets lITsIl and brain functional networks ||T6[ 171. Fig.[T](a-b) shows empirical 



evidence for the occurrence of hierarchical modular organization in the network of con- 
nections between cortical regions in the cat 1 18J and macaque [ 19] brains obtained from 
anatomical studies. Theoretical understanding of such systems is still at an early stage. A 
simple-minded deterministic construction approach towards such networks claimed that 
such structures have a specific signature in the power law dependence of the clustering 
coefficient of nodes having degree k on k 1I20I1 . We demonstrated, using simple counter- 
examples, that this is true only for a very specific set of artificially generated networks 
and does not hold in general [8]. This illustrates the importance of acquiring a clearer 
understanding of the properties that the hierarchical nesting of modular structures impart 
to a system. We focus specifically on the role that such mesoscopic organization plays in 
synchronization dynamics on the network. 

In this paper we show that in a hierarchical modular network of oscillators arranged 
into I = hiev levels, there exist hiev distinct time-scales of the synchronization dynam- 
ics. The oscillators belonging to the elementary modules (I = 0) synchronize earliest, 
followed by those belonging to the same meta-module (/ = 1) and so on, with global 
synchronization being achieved lastQ. In the next section we discuss the simple network 



' Note that if the synchronization time diverges rapidly, we may not observe synchronization beyond a certain 
level within a reasonable period of observation and thus, global synchronization may never be achieved. 
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model with hierarchically nested modules which we have used. Our results on the syn- 
chronization dynamics on such a network are described in Section 3. We have carried out 
spectral analysis of the Laplacian matrix for the network to give a theoretical background 
for the explicit numerical simulations. We conclude with a short discussion on the pos- 
sible functional relevance for the separation of time-scales through hierarchical modular 
organization in the biological world. 

2. The Model 

We have used a general model for networks having modular organization at several hier- 
archically arranged levels that has been introduced in Ref. 

f^] [Fig. [11(c)]. At the lowest 

level {I = 0), the network consists of M modules, each containing n nodes. The con- 
nectivity, or, the probability of a link, between a pair of nodes is much higher for nodes 
belonging to the same module compared to those belonging to different modules. Let us 
assume the connectivity within each of the modules to be < pi < 1. We now introduce 
the next hierarchical level {I — 1) by grouping together m modules into "meta-modules". 
The connection density between nodes belonging to different modules in the same meta- 
module is p2(< Pi)- Thus, while the oscillators in the different modules within each of 
the M/m meta-modules have a lower probability of connection (^2) between them as 
compared to the probability of connections between oscillators in the same module (pi), 
it is higher than the connectivity for oscillators belonging to different meta-modules. To 
increase the number of hierarchical levels to three, we group q such meta-modules into 
"meta-meta-modules" {I — 2). The connectivity between oscillators belonging to dif- 
ferent meta-modules but in the same meta-meta-module is p^{< p2)- This procedure of 
grouping together q clusters of oscillators at higher and higher levels of organization can 
be continued upto a maximum number of layers, hi^v, implicitly given by the relation 
qhicv-i^ = M. The parameter q is the branching at each level (except the lowest one), 
corresponding to the number of clusters at one level that are grouped together to form the 
cluster at the next higher level. 

To reduce the number of model parameters, we further assume that the connectivities 
within and between clusters at different levels, pi, p2, ■ ■ ■ , Phi^^ are related as: 

Pl=Pl = ... = ^h^=r, (1) 

Pi Pi P/l,ev-l 

where < r < 1 is the ratio of inter-cluster connectivities at two successive hierarchi- 
cal levels. Varying r from to 1, one obtains a whole range of hierarchical modular 
networks, with the special case of M isolated modules at one extreme (r = 0) and a 
homogeneous Erdos-Renyi random network at the other (r = 1). Figure [T](c) shows an 
example of a hierarchical network with 3 levels and r = 0.05 constructed following the 
above procedure. 

The model allows the modular character of the network to be changed (by varying 
r) independent of the hierarchical complexity which is governed by the number of levels, 
hiev The stochastic construction process of the network implies that reasonable statistical 
averages can be done over an ensemble of random networks with the same hierarchical 
modularity. 

3. Results 

Most networks occurring in nature have dynamics associated with their nodes fl^. In 
particular, many systems comprise oscillating elements, such as, pancreatic beta cells. 
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Figure 1. The adjacency matrix corresponding to the network of cortico-cortical con- 
nections in the (a) cat and (b) macaque brains. The largest-scale modules are indicated 
by dotted lines and labeled with Roman numerals, (c) Adjacency matrix for the hi- 
erarchical modular network of A'^ = 1024 nodes with average degree (fc). There are 
hiev ~ 3 hierarchical levels, with a branching ratio of g = 4 at each level. The ratio of 
connection density between two successive hierarchical levels, r — 0.05. The modules 
at each hierarchical level are indicated by broken lines. 
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neurons or cardiac pacemaker cells I22l 12311 . Synchronization of the periodic activity 



exhibited by the nodes in such networks may play a vital functional role. To observe how 
the hierarchical modular organization of a network can affect its synchronization behavior, 
we consider a population of N coupled oscillators. The time-evolution of the phase of the 
«-th oscillator having a frequency LOi and which is connected to ki other oscillators, can 



be described by 112411 : 



dO 1 ^ 

^ = + ^ K,, sin(0, -9,). (2) 

The coupling between the different oscillators is described by the matrix K = {Kij} = 
kA, where k(= 1) is the strength of interaction between the oscillators and A is the 
network adjacency matrix, i.e., Aij = 1 if nodes i,j are connected, and otherwise. 
In our analysis, we will mostly consider the simplified case of oscillators with identical 
frequency, i.e., LOi — lo. However, we have explicitly verified that randomly varying 
the frequencies of different oscillators over a small range do not qualitatively affect our 
results. 

The attractor for the network dynamics described by Eq. |2] is the fully synchronized 
state 9i = 6, Vi. However, for a network with mesoscopic organization, the conver- 
gence to global synchronization occurs in a series of steps which is intrinsically related 
to the underlying connection topology (and has been seen earlier in simple modular net- 



works II25I1 ). We observe that, starting from a random distribution of initial phases for the 
different oscillators, the nodes within each module synchronize first, followed by larger 
and larger structures at the higher hierarchical levels in sequence till the entire system 
oscillates in the same phase. The distinct time-scales of synchronization for the different 
levels reflect the structural organization of the network. 

To analyze the time evolution of the synchronization process, we measure a local order 
parameter using the pair-correlation function between the oscillator phases: 

p,j{t) = {cos[0,{t)-9,m, (3) 

where (. . .} is an average over random initial phases. Introducing a threshold T, the cor- 
relation matrix is converted into a dynamic connectivity matrix I?t(T), where Vij = 1 
if pij > T and = otherwise. The number of distinct synchronized communities at 
any given time is given by the number of disconnected clusters in T). Initially, when the 
phases of the oscillators are chosen from a random distribution, none of the elements are 
synchronized and the number of clusters is trivially equal to the number of oscillators, 
N . However, with time, more and more oscillators get successively synchronized and 
when global synchronization is achieved, there is only a single cluster of synchronized 
elements. In contrast to homogeneous random networks, where there is a smooth, contin- 
uous transition to global synchronization with time, for a hierarchical modular network 
we observe a series of step-like transitions as larger and larger clusters get synchronized 
at different time-scales (Fig.|2]). 

For a network having /i;eu hierarchical levels, the evolution to global synchronization 
exhibits hi^v intermediate time-scales as reflected by the occurrence of hiev plateaus be- 
tween N and 1 for the number of synchronized clusters (Fig. |2]l. First, at the relatively 
short time-scale of t„i, disconnected clusters are observed to form in V which correspond 
to the modules at the lowest hierarchical level of the network. Thus, local synchronization 
within each module at level / = is achieved relatively rapidly. Next, at a slightly longer 
time-scale of Tmm, several of the clusters mentioned earlier merge into bigger clusters 
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Figure 2. Time-evolution of tiie number of synchronized clusters of oscillators for 
hierarchical modular network with A'^ = 1024 nodes of average degree (k) = 14. 
The synchronization occurs over three distinct time-scales, reflecting the number of 
hierarchical levels hi^^ — 3 of the network. First, local synchronization occurs by 
T ~ 20 among the oscillators within each of the 64 modules (having 16 oscillators 
each) at the lowest level, Z = 0. By T ~ 80, groups of four modules (corresponding 
to a total of 64 oscillators each), which form the meta-module in the next hierarchical 
level (I — 1) get synchronized. By T ~ 720, synchronization is observed at the level 
I — 2, where each of the synchronized groups comprise 256 oscillators. Finally global 
synchronization among all 1024 oscillators occurs at T ~ 2500. The branching at 
each level is q = 4 and the ratio of intermodular connections between two successive 
hierarchical levels, r — 0.03. 
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Figure 3. The variation of thie synchironization time-scales at different liierarchiical lev- 
els as a function of r for a network having N = ^12 oscillators arranged into modules 
in hiev ~ 2 levels. The time t™ is that required for each module to synchronize the 
32 oscillators that each of them have, while Tmm is the time required for synchroniza- 
tion within a meta-module comprising four of the aforementioned modules and Tg is 
the global synchronization time (i.e., the entire system comprising four meta-modules). 
The three time-scales diverge sharply as the modularity at each level becomes more 
prominent with decreasing r. Conversely, as the network becomes more homogeneous 
when r — s- 1 the three time-scales converge. The average number of connections among 
the oscillators (k) — 30 and the branching at each level is g = 4. 



corresponding to the meta-modules at the next hierarchical level / = 1 of the network. 
Therefore, the intermediate-scale synchronization over meta-modules is a slower process 
than the local synchronization seen within each module, the initially synchronized clusters 
remaining fairly stable in the intervening time-period between r,„ and Tmm- Successive 
hierarchical levels take longer and longer times to synchronize, and we can associate dis- 
tinct synchronization time-scales r with each such level. Finally, global synchronization 
is observed at the longest time-scale of Tg. 

As the existence of the distinct time-scales to synchronization is a result of the meso- 
scopic organization of the network, we expect that as the system becomes more homoge- 
neous on increasing r, the different time-scales should converge towards the same value. 
This is indeed observed in Fig.[3j where the three time-scales for hiev = 2, viz., r^, Tmm 
and Tg approach each other as r — > 1. 

The existence of the distinct time-scales in a hierarchical modular network can be un- 
derstood by analyzing the linearized dynamics around the synchronized state of the sys- 
tem: 

where L is the Laplacian of the network, with La — ki and Lij — —Aij for i ^ j. 
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Figure 4. (a) The rank-ordered reciprocal eigenvalues for the Laplacian matrix of a 
hierarchical modular random network {hiev = 3) with A'^ = 1024 oscillators having 
(fc) = 14 connections (on average) for r = 0.05. The existence of three distinct 
spectral gaps reflects the three hierarchical levels in which the modular organization 
is arranged, (b) The width of the three Laplacian spectral gaps shown as a function 
of r for a hierarchical modular network with = 1024 oscillators having (fc) = 60 
connections (on average) arranged into hu^ = 3 levels. The spectral gap for each 
level increases with decreasing r, implying that as the hierarchical modular character 
of the network becomes more prominent, the synchronization times for the different 
levels deviate significantly. The divergence of the gaps for sufficiently low values of r 
indicate that the system may not synchronize globally for very weak coupling among 
the clusters of modules. For all cases, the branching at each level is g = 4. 



Solving Eq.|4]in terms of the normal modes (t>i{t), we obtain 



0,(0 ^.^zWe-^-*, z= l,...,iV, 



(5) 



where are the eigenvalues of L' = D^^ L with D being a diagonal matrix having 
Da = ki. All the eigenvalues of L' are real as the matrix is related to the symmetric 
normalized Laplacian D^/^ L' D^^/^ through a similarity transformation. Any difference 
in time-scales of the different modes is manifested as gaps in the spectrum of this Lapla- 
cian, and reveals different levels of mesoscopic organization in the network. The mode 
corresponding to the smallest eigenvalue is associated with global synchronization. The 
other modes provide information about collective behavior within different groups of os- 
cillators. Typically, the size of the clusters considered decrease with increasing value of 
the corresponding eigenvalues. 

In Fig. ID (a) we observe multiple gaps in the Laplacian spectrum for a hierarchical 
network, the number of gaps corresponding exactly to the number of hierarchical levels 
{hiev = 3) indicating that the existence of several distinct time-scales for the dynamical 
system has its origin in the hierarchical modular nature. As seen in Fig.|4](b), the width 
of the gaps increase with decreasing value of r, suggesting that the time-scales become 
more differentiated with increasing modularity, i.e., decreasing connectivity between the 
modules, between the meta-modules, and so on. 
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4. Discussion 

In this paper we have generalized our earlier result for time-scale separation of dynam- 
ics on simple modular networks by showing that for a system with hierarchically nested 
modules occurring at multiple levels, there exists a number of distinct time-scales (equal 
to the number of hierarchical levels). Any collective dynamical process taking place on 
such a system (e.g., coordination of nodal activity) will occur at many different temporal 
rates, the fastest events taking place at the lowest modular level and the slowest occurring 
at the global level. An important point to note is that the non-uniform synchronization 
observed in our system is rooted in its non-trivial mesoscopic organization. This distin- 
guishes the behavior from the "hierarchical" synchronization observed in homogeneous 
scale-free networks, where oscillators having more connections (i.e., higher degree) show 
stronger phase-locking between them |26]. Thus, the latter phenomenon is related to the 
dispersion of microscopic properties (viz., degree) of the nodes, which by design is iden- 
tical for all oscillators in our system. An interesting topic for future study is the effect 
of different varieties of degree distribution on synchronization dynamics in hierarchical 
modular networks. 

Our results also have ramifications for the evolution of such mesoscopic organization 
in natural systems such as the brain. While synchronization of activity among elements 
belonging to the same cluster may have functional relevance, global synchronization of 



the system is often pathological and detrimental to the system 0271 12811 . Thus, the oc- 
currence of a hierarchically nested modular organization may be nature's way to allow 
rapid local synchronization at small scales while making it difficult for activity in the sys- 
tem as a whole to get coordinated. Note that this possible functional role of hierarchical 
modularity in the brain is distinct from the recent suggestion that it is related to the task 
of simultaneously segregating sensory signals and integrating information from multiple 
channels in the brain, as the latter requires the existence of hubs (i.e., nodes having a much 
higher number of connections than the average, which by design is absent in our model) 
in addition to the modular and hierarchical nature of cortico-cortical connections [29,1 . We 
have earlier demonstrated that the occurrence of modularity in the C. elegans nervous sys- 
tem may be directly related to the functional requirement of information processing |30t]. 
A recent study of Hopfield-like associative memory models has also shown that modular 
network structure alters the attractor landscape of convergence dynamics in such systems, 
making the recall of stored patterns more efficient [31]. Generalizing these results to 
modules at more than one level should bring us closer to answering the question as to 
why hierarchical modularity is ubiquitous in the biological world. 
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